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ABSTRACT 

The arguments for the monitoring and evaluation of 
adult literacy and basic education programs are no different from 
those justifying monitoring and evaluation of any other educational 
program. The priority and status accorded different programs are 
likely to determine commitment to their internal and external 
evaluation and the form that these take. Explanations for not 
identifying common procedures for monitoring and evaluation are 
expressed in terms of complexities at program and project levels and 
imply limitations of evaluative procedures. However, common 
procedures are conceivable for monitoring and evaluation of other, 
equally diverse, educational projects and programs. Therefore, it is 
also technically conceivable in this case. Explanations for agency 
and government failure to introduce systematic monitoring and 
evaluation procedures include lack of awareness of the relevance of 
monitoring for effective project execution and of the importance of 
evaluation to future policy formulation. Structural constraints most 
likely explain the infrequency and lack of system in monitoring and 
evaluation procedures. Evaluation schemes to date have included 
assessment of the acquisition of the core skills of literacy and 
numeracy and required careful planning. A proposed common approach to 
evaluation would be a process by which participants have a part in 
systematically reviewing all stages of a .literacy or basic education 
project and its contribution to wider sectoral objectives. (Contains 
32 references.) (YLB) 
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THE MONITORING AND EVALUATION OF ADULT LITERACY AND BASIC EDUCATION 

PROGRAMMES: 
TOWARDS AN INTERACTIVE APPROACH 



INTRODUCTION 

In the 1960s and 1970s there were prodigious efforts by age^cie^s 
such as Unesco and International Development Research Centre (IDRC) to 
promote research on literacy and basic education programmes for 
adults. Recurrent themes in recent writing are the paucity of such 
research at programme and project levels and its failure to provide 
information for future policy about good practice and outcomes 
(Wagner, 1987) • In the documentation associated with International 
Literacy Year (ILY), including the World Declaration on Education for 
All and the Framework for Action to Meet Basic Learning Needs, is a 
demand for the effective monitoring and evaluation of what the 
Director of Unicef described as learning achievement and system 
performance (Grant, 1990). 

Monitoring and evaluation are intrinsic to projects and critical 
to their transformation into sustainable activity (Imboden, 1983; 
GTZ, 1990). Without them, there is little chance of this occurring. 
With them, if well-managed, is the opportunity to improve project 
implementation and, if appropriate, ensure continuity. 

There seems to be a case for encouraging programme sponsors to 
plan and use monitoring systems that are not intimidating. They should 
provide clear information to those concerned with individual projects, 
in such a way that it is accessible to others implementing similar 
schemes. To this end, it is argued, common approaches to monitoring 
and evaluation would be useful. 

Prerequisites 

Educational programmes and projects, even for adults, are not 
new, nor is there anything new about the principle of monitoring and 
evaluating these or any other such schemes* Therefore, before 
embarking on an exercise to propose procedures for monitoring and 
evaluation that will be applicable across a range of literacy and 
basic education projects and programmes, in different places and 
sponsored by different organisations, a series of questions have to be 
addressed. Three are critical. 

If there is a need for such procedures: 

1. Why has none already become part of common practice? 

2. What would they achieve beyond that already achieved by such 
systems currently applied? 

3. How would it be possible to ensure the widespread adoption of any 
new system to be developed and found appropriate? 

Outline 

Some answers to these questions emerge in the sections below: 

I. The case for common procedures for the evaluation of literacy and 
basic education projects and programmes for adults. 
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II. The failure to adopt such procedures to date. 

III. Monitoring and evaluation practices in respect of adult and basic 
education schemes. 

IV. Projects, progranunes and policies: towards a common system for 
evaluation* 

* 

V. Implications for the adoption of such schemes. 



Terminology 

Within development planning, sectoral activities are referred 
to as programmes. Programmes comprise multiple projects, the purpose 
of each one of which is to make an explicit contribution to the 
fulfilment of programme objectives. 

Monitoring refers to the systematic and critical observation of 
project and programme activities as they are being implemented to make 
a formative evaluation of the project in question. (It should be noted 
that monitoring and evaluations are themselves included among project 
activities). The forjuative evaluation makes recommendations for 
changes to enhance the further execution of the project. The final or 
summative evaluation of a scheme includes a review of the processes of 
its implementation and an analysis of their individual and combined 
effects. 

It may appear that the processes, system and overall structure of 
monitoring and evaluation are conceptually distinct from the 
activities they appraise. However, as with any investigation, the 
outcome is dependent on the methods used to obtain information. This 
means that process and product are inseparable and implies the need 
for careful understanding of what is meant by terms such as common 
procedures and standardisation. 

By literacy is conventionally understood the ability to 
communicate through the written word in a range of situations and 
numeracy the ability to apply at least simple calculations using the 
four functions. Basic education implies the development of these and 
other skills to levels sufficient for independent use as required. 
More precise, internationally accepted definitions are not agreed. If 
anything diversification of the modes of graphic representation of 
language makes a common definition more remote, as does the growing 
recognition of literacy as a symptom of processes of social change. 



I. THE CASE FOR COMMON PROCEDURE S FOR THE EVALUATION QF LITERACY AND 
BASIC EDUCATION PRQ.7ECTS AND PROGRAMMES FOR ADUT^TR 

Rationalfifi and levels of unnil-nnng and evaluation 
tOJL educational projects 

Comprehensive project monitoring should be designed to provide 
information to those concerned, at whatever level, with project design 
and implementation. This information should refer laterally to their 



own and peer performance and, hierarchically, to the ways in which it 
is affected by the performance of others and to its effect on other 
people's performance* 

In educational projects, such project level assessment of 
learning procedures informs students of their progress and teachers of 
their effectiveness, in respect of the items measured. Depending on 
results and explanations for them, such assessment serves in the first 
instance to motivate or discourage those affected. Teachers and 
learners performing well, according to their own criteria, will feel 
encouraged to continue and perhaps make greater efforts. Those 
performing less well, but identifying reasons for this upon which they 
can act, may also be motivated to continue and improve their 
performance in the future. Assessment can also be threatening, when 
teachers and learners performing at what they perceive to be below 
minimally satisfactory levels, without seeing ways in which to 
overcome their difficulties, may be sufficiently discouraged to 
withdraw from the scheme. 

Administrators accountable to sponsoring agencies and 
governments, concerned to sustain project viability, also require 
this and other information about project performance, as well as 
related explanations. With appraisals of the quality of their own 
contributions, derived from teachers and learners and from 
organisational reviews conducted by national or regional level 
representatives, they can then plan action to change aspects of 
project delivery in such a way as to increase and sustain participant 
motivation and improve effectiveness. On the flip side, a negative 
appraisal can be so undermining as to kill incentive. 

Educational project monitoring is concerned with the extent of 
skill acquisition by learners and, even at the early stages, with the 
effect of course participation on their lives. It appraises the 
quality and availability of project resources, as well as the 
organisational capacity required for their delivery. These would 
include: assessments of teacher competence and pre-service and in- 
service training; reviews of the appropriateness of teaching methods 
and material aids; analyses of the efficiency and effectiveness of 
administrative support (timely delivery of materials, payment of 
staff, responsiveness to requests for assistance, regularity of visits 
to centres, etc). Response to monitoring reports would be strategy to 
overcoir«e defects in implementation. 

The summative evaluation of educational projects takes account of 
monitoring and formative recommendations to measure outcomes in terms 
of stated project purposes. These include skill levels attained by the 
end of the course and the extent to which they are being used by 
participants immediately and after a period of time. If the time lapse 
is long enough, such analysis might attempt to measure the social and 
economic effects on local development of the use of skills derived 
from the course. The analysis seeks to explain these outcomes in terms 
cf the effectiveness of project design and implementation strategies 
so that account might be taken of critical factors, positive and 
negative, which might influence the quality and effectiveness of 
future schemes. Such evaluation also seeks to appraise foreseen 
secondary effects of project intervention and to account for effects 
that were not foreseen. 



Monitoring and evaluation in formal and non-formal education 
Monitoring pupil performance in schools, through end-of -module 
exercises and examinations, is common practice. In countries with 
centralised curricula, the assessment exercises will be the same from 
school to school, as will any end-of --school , public examinations. The 
purpose of course exercises and tests is to consolidate learning and 
grade children according to performance, usually with serious 
consequences for their after-school careers. In spite of the 
regularity of this monitoring which, to a degree, may have 
standardised performance indicators within countries, there are no 
common international monitoring systems in respect of different levels 
of schooling* If there is an assumption that the completion of primary 
schooling in one country implies the acquisition of the same 
substantive and cognitive skills as in an other, there are no measures 
with which to confirm this. The notion does not obtain at post-primary 
levels, although the increasing popularity of the International 
Baccalaureat would indicate a demand for this, at least among elite 
groups with expectations that their children will follow international 
careers . 

Typically, within countries, there are no attempts made to 
measure systematically the inter-relationship of affective influences 
and attainment within formal systems. The complexity of accounting for 
the influence of all process inputs and the notorious difficulty of 
measuring the effects of schooling on children's lives and on society 
as a whole, means that there are no indicators or procedures for this 
that are valid either across systems or cross-nationally. Instead, 
there is a marked tendency to use surrogate indicators which relate 
more to access than to outcomes. 

The case for introducing common procedures for monitoring and 
evaluation in respect of non- formal education schemes, is logically 
the same as that of formal schooling. However, the incidence of either 
formative or summative evaluation is inconsistent. Endeavours to 
identify replicable procedures have been rare and, when attempted, 
they have been complex. Skill teaching tailored to meet the needs of 
individual client groups and sometimes of individual members within 
groups, have been one factor said to inhibit the development of 
common indicators or systems. Another, probably more important factor, 
has been the varied and voluntary nature of participation in 
programmes that are outside formal systems and their frequent lack of 
relevance to competitive labour market entry. The fact that many such 
projects are strapped for cash may also be significant. 

Assumptions ri^lating to common prn cedures 

So far then, there is an assumption that there is an established 
conceptual framework of procedures for the monitoring and evaluation 
of educational development. Under certain conditions, notably in 
formal education, it is possible to identify standardised indicators 
of performance in the substantive element of a programme. It excludes 
measure of attitudinal change. 

Feasibility 

With careful planning, it would appear feasible to introduce not 
only common procedures, also outcome measures for the monitoring and 
evaluation of substantively similar, non-formal, adult basic education 
projects if they were administered by a single agency or government 



department within a country. It might also work in countries with 
established networks for the exchange of information between 
organisations sponsoring similar projects. Between countries, there 
would seem to be little problem in the mutually beneficial sharing of 
experience between similar field projects, if they are sponsored by 
the same international agency, but there would be difficulty in 
identifying common indicators of performance. Effective in-couiitry, 
inter-organisational networks would ensure the communication of such 
information to the executives of different international agencies in 
their countries of origin. The lateral exchange of such information 
between them seems to be more problematic at this level. 

H stori ca!^. sped f i ci ty 

It would be wrong to put a case for the monitoring and evaluation 
of adult literacy and basic education, without some allusion to the 
historical context in which it is being made. It would be difficult to 
divorce the current interest in the monitoring and evaluation of adult 
literacy and basic education from the preoccupation with the 
assessment of learning in formal education that reemerged in the mid- 
1970s. This was first manifest in the US and later in the UK in, for 
example, research undertaken by the Assessment of Performance Unit 
(APU). The APU endeavoured to set attainment criteria, by subject, for 
children of different ages. 

These trends are inseparable from the demand for enterprise 
accountability in business and government that has been expressed 
since the start of the recession. This has coincided with the 
diminishing importcince of welfarism and the return to the market as 
arbiter of the distribution of social well-being. 

Moving from project and programme levels to the wider social 
context, the purpose of educational monitoring and evaluation, and so 
the form that they take, is intimately related to assiimptions about 
the purposes that will be served by the educational endeavour. Any 
change in these assumptions will affect programme development and so 
evaluation. Further, in this case, it is critical to understand the 
social and political implications, and their implications for 
monitoring and evaluation, of the different structural positions (in 
relation to the state and to the labour market) of so-called formal 
initial education for young people and the range of designated non- 
formal opportunities available to adults (see, for example, Labelle, 
1985; Bacchus, et al., 1990). Intrinsic to this is an understanding of 
the various interpretations of the interaction between language, 
literacy and numeracy and social change and their implications for 
monitoring and evaluation (see, for example. Street, 1986). 

In sum, the arguments for the monitoring and evaluation of adult 
literacy and basic education programmes are no different from those 
justifying monitoring and evaluation in terms of any other educational 
scheme. It is the priority and status accorded different programmes 
that is likely to determine commitment to their internal and external 
evaluation and the form that these take. 



II. THR FAITJIRR TO ADOPT COMMON PROrKmiRRR TO DATE 



Referring only to the case of adult literacy and basic education 
prograimnes , there are many explanations for not identifying common 
procedures for their monitoring and evaluation. They are expressed in 
terms of complexities at programme and project levels and, as such, 
imply limitations of evaluatory procedures. The most commotaly cited 
refer to inter-programme and project variations. Less commonly 
addressed is the failure of planners to develop project monitoring 
systems as part of the pre-planning and design stage of project 
development, while efforts to appraise the quality of such evaluations 
as are undertaker, and to assess their applicability to the future 
adoption of common procedures are rare* Even more unusual have been 
attempts to draw inferences for formal evaluation from the lack of 
definitional and objective clarity of many adult literacy and basic 
educational programmes. These arguably underlie many of the above 
deterrents to common evaluatory procedures. 



PROGRAMME AND PROJECT DIFFERENCES 

Philosophical specificity q£ literacy and educational 

prnyrainmeft 

Variations in the philosophy according to which programmes are 
developed is one explanation for the lack of common cross-programme 
procedures. The measures used to assess processes and outcomes of 
courses intended to impart nothing more than the techniques of 
reading, writing and numeracy, unrelated to the contexts in which they 
might be used, might be said to be different from those which should 
be used in, for example, the case of functional literacy programmes. 
In these, learners are oriented to literacy and numeracy through the 
practical application of their skills in productive environments. 
Similarly, manifest differences in the approach to monitoring and 
evaluation might be expected between these schemes and those in which 
the acquisition of skills of literacy and numeracy is part of an 
awareness raising political programme of popular education. 

Clearly, against this it would be argued that any broadly 
applicable evaluatory scheme would seek to identify common inputs, 
processes and results across projects, even when motivated by very 
di f f erent phi losophies . 

Project specificity within proarammes 

If there appear to be strong reasons for the non-adoption of 
common approaches to monitoring and evaluation procedures across 
programmes, the growing importance of contextual relevance in literacy 
and basic education teaching, as a prerequisite for sustained 
participant motivation, has meant that individual projects within a 
programme may be very different, even within the same country. During 
the period of fundamental basic education, following World War II, it 
was policy for literacy training to be used as a vehicle for the 
transmission of what was seen to be desired social and technical 
knowledge (Unesco, 1949). In the Unesco Experimental World Literacy 
Programme, 1969-1974, functional literacy projects in Iran alone were 
oriented at people expecting to work in agricultural production, 
textiles, steel manufacturing and irrigation, theoretically in 
accordance with defined local skill needs (Jones, 1988). Likewise, 
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popular education initiatives across Latin America emerge with 
community action to meet specific local needs. Each differs from the 
next in its rationale, mode of orgcuiisation and in the place of 
literacy and basic education within its sphere of activities* In 
common, their participants would claim to eschew formalism (Archer and 
Costello, 1990). The decentralisation of control of the context in 
which literacy and basic skill teaching is located, which explains the 
extent of diversification within programmes or between projects within 
them espousing the same philosophies, would appear to preclude the 
possibilities of developing common approaches of monitoring and 
evaluation procedures at the substantive level. 

The basis of this argument is that it would be ideologically and 
andragogically unsound to try, and in any C6^e not possible, for a 
performance monitoring system to restrici itself to what might be 
called the context-free measurement of participant ability to use the 
core skills of reading, writing and arithmetic. On these grounds, it 
is differences in the substantive medium in which skill training is 
embedded which are said to inhibit the development of common 
assessment procedures . 

Against this position, is the argument that the application of 
skills within a single environment does not imply dominance of those 
skills. It is their transferability to other environments which would 
indicate overall competency and which should be susceptible to 
evaluation. 



Diversity of teaching methods 

Inseparable from the variations in philosophy of literacy and 
basic education programmes is the diversity of teaching methods used 
in different programmes and in projects within them. 

Over time, the evolution of teaching and learning theories have 
combined with political and economic exigencies that favour local 
control of non- formal schemes. This has encouraged the development of 
learner-oriented, participatory approaches to teaching. The trend has 
been away from what have been described as the traditional methods of 
educational progress, based on the internal logic of cumulative skills 
in conjunction with a pattern of division between subjects and a 
separation of theory and practice (Hamadache and Martin, 1986). This 
school-type model, developed according to centralist policy, typically 
with professionally trained teachers and uniform materials, is now 
rejected in favour of more functional objective or problem-oriented 
approaches. With these, projects are developed out of an analysis of 
local needs, the purposes for which the skills are to be acquired and 
an understanding of related training and knowledge required for skill 
application. It is insufficient to teach the formalities of letter 
writing without skill in the style of address appropriate to different 
contexts or information about the mechanisms of transmission and 
response. Teaching that literacy will enable self-sufficiency and 
development should be supported by discussion and information about 
the ways in which action might be taken to this end. Closely related, 
particularly in consciousness raising approaches, is the extent to 
which the starting point of any individual's participation is the 
psycho-social dimensions of their own reality. 
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Implinations for the nrinn iples of teaching 

Just as there is no one method for selecting the substantive 
medium through which the skills of literacy and basic education shall 
be transmitted, so there is no one model which governs the principles 
according to which this medium is articulated to the deconstructed 
components of the skills that are to be acquired. Variations in 
andragogical principle (convergence, integration, diversification and 
participation would be examples) and in the processes used in the 
preparation, production and dissemination of teaching materials, would 
appear to militate against the identification of coxmnon approaches to 
monitoring and evaluation. 

Debates on methods of teaching lit eracy and numerary 
A further complexity which has implications for monitoring and 
evaluation Is the diversity of beliefs about the most effective ways 
to teach literacy and numeracy skills. Cumulative skill mastery using 
a so-called synthetic approach, based on alphabet learning, phonetics 
or syllable recognition, did not disappear with the development of 
semantically grounded analytic methods. Instead, the trend is to use 
an eclectic method which combines elements of the two approaches. This 
entails the selection of graded words, sentences and passages which 
are analysed, compared and synthesised simultaneously (Hamadache and 
Martin, 1986), Within this diversity, is the endless variation of oral 
and written exercises which are used to teach and consolidate the 
understanding of individual components and processes of language as it 
is written. Similar diversity obtains in the range of methods used in 
teaching of principles of number and the extent to which this is 
integrated in literacy work. 

Teacher iaaiiAfi 

Decisions at whatever level about the philosophy of the 
programme, the principles of teaching and learning to be endorsed and 
the practical techniques to be used have direct implications for the 
kind of people who will be chosen to facilitate such schemes and 
their preparation for this work. Again, everywhere, diversity is the 
norm. 

In nearly all countries, it is unusual for those who are to teach 
adults reading, writing and basic numeracy to be qualified adult 
educators. Most frequently, they are volunteers who may have had no 
training or only rudimentary training. In less developed parts of the 
world, they may be primary school teachers, attracted to the work by 
altruism and supplements to their pay. 

In terms of monitoring and evaluation, this has two implications. 
Firstly, those planning evaluation schemes should moderate their 
expectations of the formal competencies of teachers. Secondly, since 
at the classroom level, teachers will be critical in collecting 
information for the evaluation process, the requirements to be placed 
on them in this respect must take account of this competency and where 
possible relevant training be given. This has further implications for 
the training of trainers. 

Language 

Any proposals for developing common monitoring and evaluation 
procedures must be able to overcome the problems that would be 
. encountered by the range of languages used as the medium of 
conwiuni cation in projects. These are likely to vary not only from 
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country to country, but from one population group to another within 
the same country and, not infrequently within the same programme. 

A further complication is the extent to which account should be 
taken of learner familiarity with the language being used as the 
medium through which to acquire literacy and basic educational skills. 
Learners may be native speakers, non-native speakers or, a third 
category, non-speakers of the language. Instructors may be or not 
native speakers of the medium of communication, and they may or may 
not be speakers of learners' native tongue. 

These variations derive from cultural diversities between 
language groups. They inevitably affect interpretations of the 
learning experience in different ways. 

Definitional problems 

A final factor which impedes moves to promote common procedures 
for the monitoring of literacy and other basic adult education 
programmes is the difficulty in setting criteria w:iich indicate that a 
particular level of literacy, numeracy or other basic education has 
been attained* This is hardly surprising in the light of the diverse 
approaches to literacy and basic education that are apparent at all 
levels within and between programmes* It means that there is no 
agreement about whether to be literate implies the ability to react 
appropriately to selected course material, random published material 
(newspapers, medicine bottles, street instructions, etc) in writing or 
other behaviours. 

This definitional problem is unlikely to disappear with the 
growing media challenge to reading and writing as the dominant 
channels of mass communication. Reading and writing abilities have 
always carried connotations about social status and aspirations within 
societies, discriminating against those who have not acquired them. 
Mass preference for decoding and sometimes encoding messages through 
electronic media seems likely over time to make conventional literacy, 
based on reading and writing, more, not less, socially restrictive. It 
poses major new questions for the definition of literacy and so for 
measurement procedures related to it. 

In sum 

There is no denying this diversity of context and content 
between projects and programmes, but it would wrong to claim that it 
was unique to those involved with adult education. It certainly 
symptomises the complexity of any attempt that might be made to 
construct standard measures of process or performance in all but the 
most nearly identical of adult basic educational schemes. However, it 
is possible to conceive of common procedures for monitoring and 
evaluation of other, equally diverse, educational projects and 
programmes, or projects in general, then it is also technically 
conceivable in this case. Explanations for its non- introduction to 
date have to be sought in other arenas. 
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PLANNING AND AGENCY FAILURES 



Awareness and expertise 

There are many explanations for agency and government failure to 
introduce systematic project and programme monitoring and evaf aation 
procedures- They include a lack of awareness of the relevance of 
monitoring for effective project execution and of the importance of 
evaluation to future policy formulation, including programme and 
project planning* Lack of expertise in incorporating monitoring and 
evaluation procedures from the preplanning stage of project design and 
failure to budget the necessary resource allocations to support them 
are also common. Fear of the outcomes of such appraisal is another 
deterrent to its execution, as are the political interests in arenas 
seemingly far removed from it* 

Priorities 

However unsystematic monitoring and evaluation procedures may be, 
it is difficult to imagine any project where none were applied. 
Probably the most formal requirement in schemes developed without a 
logical planning framework, which provides at all levels for 
monitoring and evaluation, is the requirement for field or country 
officers to account for project expenditure. Such reports are likely 
to involve an appraisal of the efficiency of logistical support (needs 
assessment analysis, teacher--training and appointments, the designing 
of course structure, materials production and distribution). The 
priority accorded such organisational evaluation implies a concern 
less w th project effectiveness or effects than with agency reputation 
and re^^roduction. 

The closest these studies may come to the appraisal of short- term 
project effects is likely to be in reports compiled on the basis of 
surrogate variables. These would include old friends, such as student 
enrolment and participation and teacher attendance rates. This does 
not mean that assessment of performance is not undertaken, at least of 
student performance. However, because its outcomes are not included in 
field reports concerned with budgetary accountability and the need to 
present an image of project viability, it constitutes a process of 
semi-^formal or informal monitoring. Exercises relating to particular 
topics within a course enable learners to appraise their own 
understanding. Periodic quick tests serve the same end and provide 
teachers with feed back on learner development and, hopefully, on 
their own performance. 

More informal yet is what has been described as natural 
monitoring (Kinsey, 1983). In this people reflect on their thoughts 
and behaviour, in this case learners and teachers, in relation to the 
project or programme in question, and take action, sometimes 
unconsciously, to rectify perceived deficiencies. 

Remiiremenl- ' 

At issue then is the identification of mechanisms by which this 
informal monitoring and evaluation of different aspects of project 
performance can be combined with the organisational implementation 
appraisals and converted into more formal procedures of evaluation of 
inputs, processes and effects, without undue complexity. 
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STRUCTURAL BLOCKAGES TO THE DEVELOPMENT OF SUCH SYSTEMS IN THE 
CASE OF ADULT LITERACY AND BASIC EDUCATIONAL PROGRAMMES 



A third inhibitor to the development of formal systems of 
monitoring and evaluation relates to the purposes served by the 
programmes and their status in the eyes of government, agencies and 
participants. Non-obligatory participation, voluntary or tokenly paid 
teachers, promotion by non-government organisations, lack of official 
recognition of performance levels and non-marketable qualifications, 
individually reduce the standing of the schemes with which they are 
associated. In aggregate the effect is greater* This is confirmed by 
the limited resource allocation to adult and continuing education in 
those countries that do develop national programmes, and their early 
suppression in the facx3 of budgetary constraints. This decision may be 
taken as a result of local initiative, or at the insistence of 
external funding agencies, such as the World Bank. It is also 
confirmed by failure on the part of government and sponsoring 
organisations to act upon such evaluation is undertaken. 

Ultimately, it is likely to be these structural constraints and 
their expression in political will, or lack of it, that will explain 
the infrequency and lack of system in procedures of monitoring and 
evaluation in resi^ect of adult and basic educational programmes for 
adults. Programme level obstacles, as argued above, are common to 
formal education schemes and other projects, are of themselves 
insufficient deterrents. 



III. MONITORING AND EVALUATION PRACTICES I N RESPECT OF ADULT AND BASIC 
KPTirATTON SCHEMES 

Notwithstanding all this, there have been many approaches to the 
evaluation of literacy and basic educational prograimnes, although most 
recent researchers lament the infrequency of systematic appraisal. 
This is not a complaint directed primarily at projects in less 
developed parts of the world, but also at those set up in advanced 
industrial and post- industrial societies (Holland, 1989; Lind, 1986). 

Equivalency and competency 

From the early schemes, in the US and elsewhere, which set grade 
levels equivalent to those for reading, writing and basic numeracy in 
schools, there came a shift to the assessment of adult competency, 
involving literacy and basic education in a series of life simulating 
contexts. In the US, this included the National Assessment of 
Educational Progress (Kirsch, 1990). In neither the equivalency nor 
the competency approaches, was it deemed relevant to control for 
contextual influences on the acquisition and performance of literacy 
skills. Also, it quickly became clear that there were multiple 
problems of verisimilitude and control in attempts at evaluation of 
realistic functionality. 

Context and functionality 

This criticism was fundamental in the case of the Unesco 
Experimental World Literacy Project (EWLP) that was introduced in 6 
countries between 1965 and 1974. It was intended not only to test the 
hypothesis that education for adults had a positive effect on social 
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and economic well-being, but to construct and test procedures for 
monitoring and evaluation that would be applicable within specific 
countries and projects. The scheme was to be flexible enough to allow 
for project and context variations, (as described in section II 
above), but sufficiently compatible for cross-programme or cross- 
national comparison (Jones, 1988). 

The proposal failed for many reasons, starting with problems of 
achieving a common definition of literacy between countries and the 
difficulties in standardising evaluation procedures in different 
parts of the world. The quality of data that was collected from 
project to project and from country to country, was not sufficiently 
robust to stand such analysis. This was in part because of 
international organisation insensitivity to the implications of local 
variations in implementation and context. As with the NAEP in the US, 
criticism was also levelled at the inadequacy of the appraisal of 
functionality in the evaluation of the EWLP (Jones, 1988; Unesco, 
1976). 

When, eventually, attempts were made to systematically measure 
literacy and numeracy skills in simulated contexts, using a criterion 
sampling approach, tests were found difficult to construct and 
control, very expensive and only partially effective. Other 
conventional tests were needed to supplement the information that 
could not be gathered in the simulated test environment (Shavelson,_et 
al., 1985). A further criticism was that this form of assessment was 
unable to provide information on processes of project implementation 
and how they had affected learning. 

Participation and process 

That is not to say that all of the EWLP studies overlooked this 
dimension. In Mali, the use of a participative approach to monitoring 
the literacy and basic skills educational prograimne that was 
developed in association with a ground nut development scheme used 
ipsative and criterion referencing techniques which provided 
information to learners, teachers and administrators, including 
trainers. Booklets and forms were prepared and piloted before the 
evaluation date so that teachers were confident about their use. Open 
meetings were held to discuss the observations made about processes, 
and ways to improve them and also about outcomes, in terms of learning 
achievements and other effects (Easton, 1985). 

A substantial Hawthorne effect was observed as a consequence of 
this participatory evaluation, with significant increases in enrolment 
and retention. These waned after a time if it was felt that 
participant proposals for project implementation modification were 
being ignored, in favour of those put forward by staff. The 
triangulation of data by the project evaluation team enabled some test 
of the reliability of the information gathered from teachers and 
learners using test forms and questionnaires. As an aside, the 
project's observation of the limited scope for using the skills 
learned in the classes in community development activities and the 
resulting hypothesis about the numbers of people with literacy skills 
required by any population group should be further investigated. 
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The laborious efforts made by the EWLP to create evaluation 
systems that were applicable across countries and programmes 
identified a series of 167 indicators for the measurement of project 
performance. Of these, 11 were described as mandatory and 9 as 
recommended. They included the ratios of inscription, drop-out, 
attendance, time utilisation, skills achieved, knowledge acquired, 
skill use, change in learner productive activities, changes in use of 
consumer durables, schooling of participant children, understanding of 
health, hygiene and nutrition. 

Leaving to one side the consumer orientations of this list, which 
confirm the ideology of the EWLP, none of the indicators took account 
either of information about the process of project implementation or 
the processes of learning that had contributed to observed 
performance. Nor did they, or the other schemes to which reference has 
been made, attempt to appraise the extent to which the outcomes of 
projects are other than the acquisition of formal skills and their 
application in economically productive environments. 

Psvcho-social dynamics 

Content analysis of learner and tutor accounts of their 
experience has identified five categories of psycho-social effects of 
literacy programmes on adults in Britain, all of whom had attended 
school as children. In oral accounts they ranked as affective personal 
achievements, affective social achievements, socio-economic 
achievements, cognitive achievements and enact ive achievements. In 
written accounts cognitive achievements moved into second place 
(Charnley and Jones, 1979). 

The recognition of the need to perceive as positive both 
affective and cognitive outcomes of programmes has been accompanied 
by attempts to constructively appraise contextual processes which 
enhance project execution and learning. Learner oriented, these are at 
the levels of learners, teachers and organisations and seek to take 
account of the interaction between them (Holland, 1990). 

P rofiling 

Devices used to assess these multiple endeavours include 
standardised tests of knowledge and skill and procedures for their 
application, centre reports which, completed by teachers, provide 
basic statistics and information about centre activities. More 
recently, in an effort for students to monitor their own progress 
towards objectives that they have set themselves. Profile Progress 
packages have been prepared which it is hoped will contribute to 
sustained learner motivation (Holland, 1989, 1990). Similar check- 
lists to monitor the fulfilment of administrative and teaching 
commitments have been proposed for literacy and basic educational 
schemes administered by local government authorities in Britain 
(Wells, filL_al., 1991). It is too soon to appraise their usefulness in 
their intended environments. In less developed countries, it is 
unlikely that literacy and basic education schemes would have 
available the funds to produce such monitoring materials or that, 
beyond the centres to which they refer, there will be staff capable to 
collate information from reports in clusters of centres. This is not 
to descry the worthiness of the scheme, but to identify obstacles that 
wo lid need to be overcome in other contexts. 
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Core indicatorg 

In common to all of the evaluation schemes referred to above is 
their inclusion of assessment of the acquisition of the core skills of 
literacy and basic numeracy. This is regardless of the andragogic 
tradition which they espouse and of the variations in definitions of 
the skills that represent a state of literacy or numeracy. Exceptions 
are likely to be found in those conscientization programmes which may 
eschew formal literacy training and also, more commonly, evaluation, 
about which therefore little is known. 

A second feature, explicit in some cases, is the need for 
careful planning of any evaluation that, at some point, will require 
the transmission of information about teachers and learners and the 
interaction between them to interested parties outside their place of 
meeting. Principles to guide this planning procedure change over time. 
In fashion today are those of participation cind reinforcement, as 
reported in the Mali evaluation and strongly present in the process of 
profiling. Other principles to be endorsed are simplicity, information 
minimal i sat ion and cheapness. 

Finally, from the critiques of the EWLP has come a list of 
practical prerequisites for any evaluatory procedures intended to be 
used in comparative contexts. They are: a common set of indicators; 
the unification of evaluation terminology; similarity of formal 
characteristics of instrumentation (methods and techniques); a common 
approach to the treatment of the collected data (Jones, 1988). 

In sum 

Given that formal evaluation is a process intended to enhance 
organisational efficiency and effectiveness, its appropriateness to 
schemes that claim to reject formalism may be challenged. However, in 
this there appear to be certain contradictions. All education is 
concerned with changing the ways in which people think and behave. 
Similarly, other than informal learning, all education, including that 
labelled non formal or consciousness raising education, is realised 
through organised activities which are usually described as projects 
or programmes. As such, given the definition of what constitutes a 
project, the scope for their evaluation is the same. 

A further commonality, is the goal, as opposed to the immediate 
objectives, of programmes which appear to endorse different 
philosophies and approaches to teaching. Social and economic well- 
being, the alleviation of poverty, access to the means of subsistence 
and education are goals of most educational schemes and will be stated 
as such by participants, teachers and providers around the world. It 
is assumptions about the dynamic for such convergence of thinking and 
about the most suitable ways to achieve the goal which vary. These 
produce not variations in the types of vehicle in which, in this case, 
educational goods are transported, but in the substantive medium in 
which the skills to be acquired are embedded. Once again therefore, it 
becomes possible to see how the principles underlying the monitoring 
and evaluation of projects may be the same. 

These structural commonalities suggest, beyond the outcome 
rhetorics and myths of development and personal well-being, that all 
such schemes are fruits of the same ideology. The logic of capital 
requires alienation from non-capitalist cultures, endorsement of 
consumerism and the willingness to sell labour for income. The 



schooling of children has long been accepted as being instrumental in 
this process and now writers on adult and non-formal education are 
treating it from similar perspectives (Bacchus, at — aJU. 1990). 
Awareness of this makes it possible to accept the argument for the 
application of similar evaluatory procedures in programmes with 
seemingly different philosophies. In this, evaluation itself 
contributes to the process. 



IV. PROJECTS. PROfiRAMMEfi AND POLICIES: TOW ARDS A COMMON APPROACH TO 
EVALUATION 

We seem to have reached the point of recognising that the only 
legitimate comiaonality in approaches to the monitoring and evaluation 
of adult basic education programmes has to be at the conceptual level 
and not at the practical level of the evaluation process. At this 
level of generality, the continuing failure to adopt common models 
becomes questionable. 

Projects are normally located within clearly defined policy 
cycles. This means that plans for the monitoring and evaluation of 
project and programme design and implementation, whether administered 
by government or non-government agencies, should include the appraisal 
of their expected contribution to the fulfilment of stated sectoral 
objectives and, at a higher level, to national development goals. This 
will be achieved by aggregating the effects of the project on 
individual participants and others affected by the scheme. It will 
also to allow the appraisal of the adequacy of project resources and 
the processes associated with their use. 



Fig. 1: Logical planning latrix 
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Fig* 2 The EIPOL process* 
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Fig. 3 The EIPOL grid* 
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In theory, the careful use of a device such as a logical planning 
matrix permits the verification that this is being done. It can 
provide for the appraisal of all aspects of project design and 
implementation, including the original assessment of the needs being 
addressed and the identification of possible problems. (GTZ, 1990). 

Within such a framework a further matrix may be constructed in 
which the evaluation of various elements at different stages of 
project planning and implementation may be developed. An EIPOL grid is 
one such approach (Dave, 1980). In this, the evaluation designed at 
the pre-planning, planning, implementation and assimilation stages of 
the project accounts for the analysis of the enviroiunental setting of 
the project, inputs, processes, outcomes and long-range effects (Figs. 
2 and 3). Use of such a scheme would ensure, for example, that due 
account was taken at the preplanning stage, of factors identified by 
earlier researchers as having a significant contribution to project 
enhancement. This would include consideration of, for example, the 
appropriateness of Bellahsene's literacy project implementation 
systems (Bellahsene, 1973) and reference to Roger Convert's 
explanatory models of the demographic, social and economic dynamics of 
relations within learner groups and between them and teachers (Fig. 4, 
Convert, 197f). It might also seek to appraise the usefulness of 
Ebola's model of the interdependence of programme, evaluation and data 
systems in literacy project monitoring and evaluation (Bhola, 1979). 

Although popular with funding agencies, there are problems with 
the use of such matrices. As described, they enable coherent 
appraisals of project development in relation to plans and objectives, 
but they do not specify the structure of relationships between those 
who are to participate in the process of evaluation. They appear to be 
tools for administrators and planners and less devices which will 
enable formative recommendations during the course of project 
development to those in the field. 

In practice, the potential for design flexibility is present 
within either of the two schemes above. Nevertheless, it would seem 
beneficial to introduce a further dimension to the model which allows 
ipsative evaluation at each level, as well as systematic cross-level 
appraisal. In this way, in keeping with the contemporary concerns with 
participation, arrangements can be made to ensure communication 
between all actors in the process of project development and 
discussion and action as appropriate, in response to information 
obtained through the evaluation. 

As indicated in Fig. 5, there is scope for the planning of self 
evaluation by actors with different roles in project development. Box 
Al, for example, permits learner self appraisal. The form which this 
might take will depend on the type of project and its purpose, but it 
might include: self appraisal of cognitive skills, in terms of 
performance in exercises and tests; account of non-cognitive effects 
of learning and programme participation; identification of performance 
obstacles and enhancers. The frequency of such appraisal, beyond 
natural appraisal, would again be by agreement with other interested 
parties, teachers and administrators (including trainers). In cell B2 
teachers would appraise their own performance in terms of strategic 
planning achievements and learner responsiveness. 
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In cell h2, learners would appraise the adequacy in quantity and 
quality of teaching inputs, teacher performance and material aids for 
example, again at agreed regularity. In cell A3 they would do the same 
in respect of whatever contact they may have with programme 
administrators. Although unlikely for the majority, any contact with 
central administration would receive the same treatment in respect of 
cell A4. 

This activity would enable actors consciously to appraise their 
position in relation to those others with whom they have contact 
(along the horizontal access). It would also ensure the exchange of 
information between levels: from learners to teachers and back; from 
field administration to teachers and back and from the field to the 
centre and back. 

The way information was to be collected in each cell would vary 
from project to project, and according to the needs for information of 
participants at different levels in the process. It would without 
doubt need careful planning, to ensure that a combination of 
performance, attitudinal and explanatory data were provided (while 
respecting the principles of clarity, cost-effectiveness and minimal 
intrusion) . 

The project diversity factor makes more precise recommendations 
for standardising monitoring and evaluation procedures (such as 
performance indicators) at project level, irrelevant at this stage. A 
nesting of planning frameworks, such as that suggested, combined with 
the f lexi-framework outlined in fig 5., ensures that the general 
procedures are applicable at the programme level, in such a way that 
project planners and administrators can apply them to specific 
situations. Only after satisfactory schemes have been developed would 
there be a case for seeking to test their cross-programme/cross- 
na t i ona 1 app 1 i cab i 1 i t y . 

Cant :^ on 

There is nothing new in this kind of thinking. Further, there is 
a danger that in the bid for comprehensiveness, the implementation of 
such a scheme can become complex and saturated in data to the extent 
that clarity of observation is lost. If the scheme is to be applied, 
the planning should ensure that information about it is incorporated 
simply and clearly into the training of the trainers and passed on in 
the training of field administrators and teachers. 

The piloting of the scheme in such a way as to produce useful 
measures without mega-technical statistical complexity will also 
require creative planning according to some reliable procedure for 
evaluation project planning (Programme evaluation and review technique 
(PERT) or Gantt spring to mind (Cook, 1966; Clark, 1952). It goes with 
out saying that any such formal planning for the evaluation of 
literacy and basic education projects, should at the pre-planning 
stage be clear about its purposes and the ways in which its results 
will eventually be used. 

Benefits? 

If such procedures were piloted and found to have applicability 
across projects and programmes it would be on account of their users 
decision to provide quick and succinct information derived from all 
actors in project development: This would meet today's requirement of 
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maximal participation in decision-making anA of ensuring the multi-way 
flow of information between different spheres of implementation 
activity. To encourage the widespread adoption of the procedures it 
would be necessary, during and after pilot studies of the viability of 
the scheme, to give as much publicity as possible to its virtues. Not 
only would this encourage funding agencies to look favourably on 
projects which include it, it may mean that, in time, monitoring and 
evaluation became accepted by practitioners as intrinsic to project 
design and implementation. 



V. DISCUSSION 

Proposing a scheme by which actors have a part in 
systematically reviewing all stages of a literacy or basic education 
project and its contribution to wider sectoral objectives is consonant 
with contemporary promotion of social planning processes. In these, 
local identification of needs and initiation of action to meet them 
are seen as critical factors in the achievement of individual or 
social well-being (Conyers, 1982; Midgley, et al • 1986) • 

In theory, such participation might imply, as an ultimate goal, a 
previously unimagined state of well-being requiring hitherto unknown 
fulfilment strategies. In practice, images of well-being tend to be 
familiar. They are firmly embedded in contemporary ideology and the 
political economy from which this derives. What varies is the priority 
which different individuals or groups give at any point in time to the 
achievement of the pre- requisite components of this well-being, for 
themselves and/or for others. Simplistically, it may be irrelevant for 
a government or agency to promote literacy or basic education in a 
community currently more concerned with those parts of well-being that 
can be enhanced vzith improvements to its water supply. 

Clearly, it is conceivable that there is a structure to the 
prioritisation of organised activities intended to promote well-being. 
Within this, clusters of activities of similar priority may be 
realised at the same time, as part of a coordinated plan or randomly. 
Watersheds may be reached at which new skills have to be acquired, 
which of themselves may have little impact on well-being, to enable 
the development of more complex activities. Examples of such enabling 
skills include literacy and basic education. They also include the 
organisational capacity to plan, undertake and follow-up action 
(themselves often dependent on some educational skills). 

The logic of social planning would seem to be that the community 
can identify its needs and find the appropriate resources (locally or 
elsewhere) to fulfil them, on the assumption that such resources are 
available and can be acquired. The weakness of the argument for 
autochtonous action is that, in a context in which well-being is 
predicated upon participation in market capitalism with its 
hierarchical division of labour, action to meet a particular need 
using external resources is triggered by knowledge that those 
resources exist, because someone in the community (native or newcomer) 
has experience of them elsewhere and is actively promoting them. 
Symbiotically, the dynamic of capital in this respect is to encourage 
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such action as will increase the peripheral market. The boundary 
between doing this responsively or proactively is notoriously blurred. 

Once engaged, even in the least integrated of communities, even 
as a result of the most participatory of initiatives, even in the 
least production-oriented of activities, there is a requirement for 
resource administration and accountability. Each of these, however 
effectively performed, whether in the interests of the receiver or the 
provider, implies the application of market-oriented rationality and 
so its internal isat ion. Given the number of well-being oriented 
activities conducted simultaneously in any community, the aggregate 
effect of these experiences on thinking and behaviour will be 
considerable (Preston, 1987) • 

In all these processes, identification of the source of 
initiative (local or external) becomes impossible. Empowert^ent derives 
from understanding this impossibility and its implications for goal 
achievement . 
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